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DETAILED ACTION 

Response to Amendment/Arguments 

1 . This Office Action is in response to the Response/Amendment filed on January 
16"\ 2008. Claims 1-17 are now pending in the application. Claims 1-16 were elected to 
be prosecuted on the merits. Thus, non-elected claim 17 is withdrawn from consideration. 
In a response to this Office Action Applicant should cancel the non-elected claims to 
expedite the prosecution, should the response place the instant application in a favorable 
condition for allowance. 

Drawings 

2. Figures 1 -1 IB should be designated by a legend such as ~ Prior Art ~ because 
only that which is old is illustrated (See Part 2 Description of the Related Art , pages 1- 
16). See MPEP § 608.02(g). 

Claim Objections 

3. Claims 1-9 and 16 are objected to under 37 C.F.R 1 .75 because of the following 

formalities: 

In claims 1-9 and 16, the term "operable to" makes claim limitation(s) following 
the term optional and does not required the steps to be performed. 
Appropriate corrections are required. 
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Claim Rejections - 35 USC §103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Note: The term "operable to" makes claim limitation following the term optional and 
may not require the steps to be performed. Appropriate correction is required. 

5. Claims 1-2, 10-1 1 and 16 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Applicant's admitted prior art (see Part 2, Description of the Related 
Art, pages 1-16; hereinafter refer as 'AAPA') in view of Yasuda, Hiroshi ("International 
Standard for Multimedia Encoding", Maruzen Co., Ltd. pp. 221-232, June 30, 1991; 
hereinafter refer as 'Yasuda'). 

- In regard to claims 1,10 and 16, AAPA discloses (see Fig. 8 and in the 
respective portions of the specification) a method, computer program (wherein a program 
is obvious for performing all fimctionalities for such method and device) and multiplexer 
('conventional multiplexer' or clement 960 in fig. 8), method an that is operable to 
generate multiplexed data by midtiplexing packets of media data including image data 
and at least one of audio data and text data (for example see Part 2, Description of the 
Related Art, page 1, line 30 through page 2, line 4; page 10, lines 22-25), which 
comprises 
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a media data obtainment unit operable to obtain the media data (elements 961- 
962, 964-965 in fig. 8; for example see Part 2, Description of the Related Art, page 10, 
lines 26-30; wherein elements 961-962 are for video input data, and elements 964-965 are 
for audio input data, e.g. "media data''); 

a packet header part generation unit (element 969 in fig. 8) operable to generate 
a packet header part that holds a header of the media data on a basis of the packetization 
part (for example see Part 2, Description of the Related Art, page 12, line 31 through 
page 13, line 5); 

a packet data part generation unit (element 970 in fig. 8) operable to generate a 
packet data part that holds entity data of the media data on a basis of the packetization 
part (for example see Part 2, Description of the Related Art, page 13, lines 3-14); and 

a packetization unit (element 971 in fig. 8) operable to generate a packet by- 
connecting the packet header part with the packet data part (for example see Part 2, 
Description of the Related Art, page 13, lines 15-17). 

AAPA also discloses the first and second analysis units 963, 966, {''analysis 
unit") for analyzing the media data and obtaining information for sample in MP4, e.g. the 
smallest access unit of the image data, audio data and text data (for example see Part 2, 
Description of the Related Art, page 10, line 31 through page 11, line 12); and the 
packetization part determination unit 967 ("packetization part determination unit") for 
determining packetization part for media data (for example see Part 2, Description of the 
Related Art, page 10, line 31 through page 11, linelV) and header information for 
samples, e.g. type, size and playback time duration information ("playback start times"; 
see fig. 10; Part 2, Description of the Related Art, page 12, lines 1 1-24), for storing in 
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packet generation table store unit 968 (for example see Part 2, Description of the Related 
Art, page 12, line 25 through page 13, line 24); AAPA discloses all the claimed subject 
matter; but fails to consider about playback times for audio and video are made to be ''the 
same" or synchronizing for samples, when multiplexing media data in the conventional 
multiplexer (for example see figs 1 lA-B; Part 2, Description of the Related Art, page 13, 
line 25 through page 16, line 14). 

Yasuda teaches, from the same or similar fields of endeavor, e.g. for AV 
sjoichronization, method for multiplexing AV segments and synchronizing time duration 
between audio and video segments to corresponding "same time", through methods as 
disclosed in Sections 1 1.3.1, 1 1.3.3 of AV synchronization algorithm section, pages 5-9. 

Thus it would have been obvious to the person of ordinary skill in the art at the 
time of the invention was made to apply the known method as taught by Yasuda, e.g. AV 
synchronization, into the AAPA's system, with the motivation being to reproduce media 
data while maintaining synchronization for AV segments as disclosed in Yasuda: page 5, 
line 28 through page 6, line 3. 

- Regarding claims 2 and 1 1 , AAPA fiirther discloses, wherein the information for 
video track, audio track and text track are stored in the movie header box 917 ("leading 
part of the packetization parf; for example see fig. 3 A; Part 2, Description of the Related 
Art, page 4, lines 13-27); but fails to consider about playback times for audio, text and 
video are made to be 'Hhe same" or synchronizing for samples, when multiplexing media 
data in the conventional multiplexer (for example see figs 1 1 A-B; Part 2, Description of 
the Related Art, page 13, line 25 through page 16, line 14). 
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Yasuda teaches, from the same or similar fields of endeavor, e.g. for AV 
synchronization for complex data which comprises audio, video and other data such as 
text data (for example see Yasuda: Section 11.1 Major applications and requirements, 
pages 1-3), method for multiplexing AV segments and synchronizing time duration 
between audio and video segments to corresponding "same time", through methods as 
disclosed in Sections 1 1.3.1, 1 1.3.3 of AV synchronization algorithm section, pages 5-9. 

Thus it would have been obvious to the person of ordinary skill in the art at the 
time of the invention was made to apply the known method as taught by Yasuda, e.g. AV 
synchronization, into the AAPA's system, with the motivation being to reproduce media 
data while maintaining the time corresponding relation between AV segments as 
disclosed in Yasuda: page 5, lines 28-30. 

6. Claims 3-6, 8-9, 12-13 and 15 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Applicant's admitted prior art (see Part 2, Description of the Related 
Art, pages 1-16) in view of Yasuda, Hiroshi ("International Standard for Multimedia 
Encoding", Maruzen Co., Ltd. pp. 221-232, June 30, 1991), as applied to claims 1, 10 and 
16 in part 5 above, and ftirther in view of Moriyama et al. (U.S. 5,537,409; hereinafter 
refer as 'Moriyama'). 

- In regard to claims 3 and 4, the combination of Yasuda and AAPA does 
discloses method for synchronizing complex data which comprises audio, video and other 
data such as text data (for example see Yasuda: Section 11.1 Major applications and 
requirements, pages 1-3); but fails to expUcitly disclose the playback duration of audio 
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and text samples is after (see claim 3) or is before (see claim 4) the playback duration of 
video sample. However, such implementation is known in the art. 

Moriyama teaches, from the same or similar fields of endeavor, e.g. system and 
method for synchronizing video, data and audio signals (for example see Moriyama: 
figs. 5a-d); wherein time-sequential relation between actual video and audio information 
is leads ("is before") or lags ("is after") depending on the number of audio access unit 
'AAU' in the pack (for example see figs. 8a-e; col. 8, lines 14-53); and presentation start 
times for video and audio signals in reproducing ("the earliest to playback start time of 
image data sample"; for example see fig. 9; col. 8, line 54 through col. 9, line 5). 

Thus it would have been obvious to the person of ordinary skill in the art at the 
time of the invention was made to implement the known method as taught by Moriyama, 
into the combination of Yasuda and AAPA's system, with the motivation being to 
reproduce media data while controlling difference between presentation start times 
corresponding to positional information for AV segments as disclosed in Moriyama: col. 
2, lines 9-18. 

- Regarding claims 5 and 12, the combination of Yasuda and AAPA fiirther fails 
to explicitly disclose method for obtaining "intra frame information" indicating that the 
sample is an "intra coded sample" in determining media data. However, such 
implementation is known in the art. 

Moriyama fiirther discloses the coded frame image with types of pictures ("intra 
frame information") for I picture or Intra coded picture ("intra coded sample"), P picture, 
and B picture (see figs. 1, 2a-c) with different coding scheme in MPEG in determining 
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video data, e.g. ''''determine the media data ... obtains the intra frame infonnation", as 
disclosed in figs. 1-2; col. 2, line 62 through col. 3, line 39. 

Thus it would have been obvious to the person of ordinary skill in the art at the 
time of the invention was made to implement the known method as taught by Moriyama, 
into the combination of Yasuda and AAPA's system, with the motivation being to 
reproduce media data while reducing the delay time for decoding process as disclosed in 
Moriyama: col. 3, lines 27-30. 

- In regard to claims 6 and 13, the combination of Yasuda and AAPA further fails 
to explicitly disclose method for placing a sample of the video data including the intra 
frame information in a leading part of the packetization part. However, such 
implementation is known in the art. 

Moriyama further discloses method for placing coded frame image with types of 
pictures, e.g. "sample of the video data including the intra frame information" in each 
unit of group of pictures 'GOP', e.g. ''leading part of the packetization parf \ as disclosed 
in figs. 1-3; col. 2, line 62 through col. 3, line 39. 

Thus it would have been obvious to the person of ordinary skill in the art at the 
time of the invention was made to implement the known method as taught by Moriyama, 
into the combination of Yasuda and AAPA's system, with the motivation being to 
reproduce media data while reducing the delay time for decoding process as disclosed in 
Moriyama: col. 3, lines 27-30. 
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- Regarding claims 8-9 and 15, the combination of Yasuda and AAPA further 
fails to explicitly disclose method for interleaving samples of media data such that the 
playback start times of the samples are in an ascending order or when the previously set 
condition is satisfied. However, such implementation is known in the art. 

Moriyama further discloses method for rearrangement fi-ames ("interleaving 
samples of media data") in opposite order to one shown in fig. 2, e.g. "in an ascending 
order", when the presentation starting times for I and P pictures are lagged as disclosed in 
col. 4, lines 23-27; or when the previously set condition is satisfied (for example see fig. 
3; col. 2, lines 1-8; col. 4, lines 30-38) 

Thus it would have been obvious to the person of ordinary skill in the art at the 
time of the invention was made to implement the known method as taught by Moriyama, 
into the combination of Yasuda and AAPA's system, with the motivation being to 
reproduce media data while controlling the difference between presentation start times 
corresponding to positional information as disclosed in Moriyama: col. 2, lines 1-18. 

Allowable Subject Matter 

7. Claims 7 and 1 4 are objected to as being dependent upon a rejected base claim, 
but would be allowable if rewritten in independent form including all of the limitations of 
the base claim and any intervening claims. 

Conclusion 

8. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 
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Sawada, Hideki (U.S.6,5 12,884), Murakami et al. (U.S.7,054,546) and 
Kashiwagi et al. (U.S. 5, 923, 869) are all cited to show devices and methods for 
improving the synchronization of media data such as video, audio or text, in reproduction 
media data, which are considered pertinent to the claimed invention. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Tri H. Phan, whose telephone number is (571) 272-3074. 
The examiner can normally be reached on M-F (8:00-4:30). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Chi H. Pham can be reached on (571) 272-3179. 

Any response to this action sliould be mailed to: 

Commissioner of Patents and Trademarks 

Washington, D.C. 20231 

or faxed to: 

(571) 273-8300 

Hand-delivered responses should be brought to Randolph Building, 401 Dulany 
Sfreet, Alexandria, VA 22314. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the Technology Center 2600 Customer Service Office, 
whose telephone number is (571) 272-2600. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
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published applications may be obtained from either Private PAIR or Public PAIR. Status 
information for unpublished applications is available through Private PAIR only. For 
more information about the PAIR system, see http://pair-direct.uspto.gov. Should you 
have questions on access to the Private PAIR system, contact the Electronic Business 
Center (EBC) at 866-217-9197 (toll-free). 

/Chi H Pham/ 

Supervisory Patent Examiner, Art Unit 

2616 

5/1/08 



/Tri H. Phan/ 
May 5, 2008 



